Skip to content

correct copysign backward #74322

Merged
lshpku merged 2 commits intoPaddlePaddle:developfrom
wangyuwen1999:develop
Aug 11, 2025
Merged

correct copysign backward #74322
lshpku merged 2 commits intoPaddlePaddle:developfrom
wangyuwen1999:develop

Conversation

@wangyuwen1999
Copy link
Contributor

@wangyuwen1999 wangyuwen1999 commented Jul 30, 2025

PR Category

Operator Mechanism

PR Types

Bug fixes

Description

paddle.copysign对于大tensor的问题修复

  • copysign kernel在进行float16的[x,y]反向时调用的求导公式先进行了dout * copysign_func(x, y),在copysign_func(x, y)结果为6e-8,dout < 0时,乘积下溢,导致结果为0,出现错误。
  • 现通过先进行copysign_func(x, y)和x的相除,得到结果为±1,避免下溢。

@paddle-bot
Copy link

paddle-bot bot commented Jul 30, 2025

你的PR提交成功,感谢你对开源项目的贡献!
请关注后续CI自动化测试结果,详情请参考Paddle-CI手册
Your PR has been submitted. Thanks for your contribution!
Please wait for the result of CI firstly. See Paddle CI Manual for details.

@paddle-bot paddle-bot bot added the contributor External developers label Jul 30, 2025
if (x == static_cast<InT>(0))
outs[0] = static_cast<OutT>(0);
else
outs[0] = static_cast<OutT>(dout * (funcs::copysign_func(x, y)) / x);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这样改有依据吗?比如看下torch是这样改的吗?我怎么感觉这样改虽然解决了下溢问题但可能带来新的精度问题

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

这样修改的一个依据是这个kernel的原作者针对copysign的反向写了3个GradFunctor,在CopySignGradXFunctor和CopySignGradXYFunctor中,对于x的求导顺序出现了差别,说明是作者在编程时错写了CopySignGradXYFunctor针对X的求导,将右括号放错了位置。
0d15166e28aa20c32dfbb2231387601c
当然,我们可以参考torch的实现。
383a79de6fc6ddd98f7f7e64e34a722b
同样是先除再乘。

@wangyuwen1999 wangyuwen1999 changed the title correct copysign backward correct copysign backward && std backward Aug 8, 2025
@wangyuwen1999 wangyuwen1999 changed the title correct copysign backward && std backward correct copysign backward Aug 8, 2025
Copy link
Contributor

@wanghuancoder wanghuancoder left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@lshpku lshpku merged commit 7072535 into PaddlePaddle:develop Aug 11, 2025
102 of 105 checks passed
Enigmatisms pushed a commit to Enigmatisms/Paddle that referenced this pull request Aug 11, 2025
* correct copysign backward

* correct codestyle
maxiaolong001 pushed a commit to maxiaolong001/Paddle that referenced this pull request Aug 12, 2025
* correct copysign backward

* correct codestyle
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor External developers

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants